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Executive summary 


This rapid evidence assessment aimed to summarise the efficacy of remote professional development (PD) 
approaches to support school leaders and PD providers with decisions they are making about PD provision given 
social distancing requirements caused by the 2020 Covid-19 pandemic. 


The review systematically searched for meta-analyses and systematic reviews to find the best evidence for fully 
remote and for blended PD approaches (key definitions of bolded terms are provided in Table 1 below). 


The research questions posed were: 


1. What evidence is available on the effectiveness of remote professional development approaches, compared 
to face-to-face, blended approaches and other remote approaches? 


2. a) Does some face-to-face contact moderate outcomes in remote professional development approaches? 
b) How do different remote professional development approaches moderate outcomes? 


3. What can we learn from existing systematic reviews and meta-analyses about the characteristics of effective 
remote professional development implementation? 


Limited evidence was discovered during the rapid review process. The review is structured to reflect the themes in the 
available literature. This report summarises the findings from 17 systematic reviews and meta-analyses answers 
under 5 key topic areas: 


- Effectiveness of remote PD (RQ1 and RQ2a) 

- Synchronous and asynchronous remote PD (RQ2b) 

- Remote coaching, mentoring and expert support (RQ2b) 
- Evidence on common approaches (RQ2b) 

- Support from leadership teams (RQ3) 


In each section, we have detailed the included reviews and listed findings and implications drawn from the evidence. 
As findings were drawn from education,welfare and public health professions, we have separated out the findings for 
professionals in schools in each section. Where relevant, we have also highlighted case studies from projects funded 
by the Education Endowment Foundation in which similar approaches have been rigorously evaluated in English 
schools. 


Key findings and implications 
1. Professional development can be supported effectively remotely 


School professionals can gain knowledge and skills through remote PD, ultimately leading to gains in pupil 
outcomes. 


Evidence is mixed as to whether remotely delivered PD is more or less effective than face-to-face PD, but specific 
benefits include lower costs and time incurred through travel. Other design principles are likely to be more important 
to PD outcomes than whether delivery is face-to-face or remote. There is no strong evidence that school-based PD 
should be delayed until it can be delivered face to face. 


Blending synchronous and asynchronous delivery and blending face-to-face and remote learning may offer specific 
benefits by enabling trainees to feel part of a community while retaining travel cost savings, flexibility and social 
distance during remote and asynchronous elements. 


2. Remote coaching, mentoring and expert support can be effective alone or as part of broader PD 
programmes 


Coaching and mentoring can improve skills and knowledge of professionals when delivered remotely and may 
reduce feelings of isolation in professionals. 


Remote or blended coaching, mentoring and expert support can be used to complement broader remote or 
blended PD programmes. 


3. The use of video can enhance remote PD 


The use of video is identified across a number of reviews as a particularly effective element of PD that enables 
teaching staff to review their own and reflect on others’ actions in the classroom. The targeted use of videos 
generally increases the time trainees take to complete PD but is associated with gains in practitioner knowledge 
and pupil outcomes, suggesting the additional time might be spent engaging more closely with the content. 


Video viewing is unlikely to be impactful in isolation and should instead be paired with other learning resources 
such as viewing guides or discussion with other professionals. One contradictory finding (though not limited to 
remote PD) suggests caution and monitoring is necessary when using video resources to support coaching 
conversations. 


4. Interactive content and opportunities for collaboration hold promise for remote professional 
development 


More interactive content tends to increase the time practitioners take to complete PD and is associated with better 
completion rates, knowledge and skill acquisition. Soaced education approaches (such as regular email surveys) 
could be particularly promising and may offer a low-cost way of enabling ongoing interaction with PD content. 
Conversely, barriers such as information being difficult to access can have a detrimental effect on user 
engagement. 


Collaboration between colleagues may also improve PD outcomes through enabling reflective practice and 
collective problem-solving. For example, PD providers may include peer small-group discussion sessions 
following completion of individual tasks. 


5. Remote professional development requires supportive school conditions (support from leaders, 
protected time, tech-specific training, platform ease of access) 


School leaders have a critical role to play in ensuring enabling conditions are provided for remote PD to be 
successful. They can support staff to prioritise their PD by creating protected time within the working day for staff 
to engage with PD sessions or materials. 


Schools should ensure staff have access to technology required for their PD and appropriate training in order to 
access this safely, efficiently and appropriately. PD outcomes are strengthened where the purpose of the PD, 
roles and expectations are clear; this may require co-ordination from the PD provider, trainee, and school 
leadership team. 


Limitations 


It is important to note the limitations of applying existing evidence to the current context. None of the 

studies examined measure impacts of remote PD during a global pandemic, and there are characteristics of the 
current situation that will be unique: the demands of responding to learning loss for many children, some teachers 
continuing to work from home, staff illness and other restrictions related to social distancing, and the possibility of 
further school closures. 


In addition, there are other important limitations to the evidence. Few high-quality reviews have looked at remote PD 
so the evidence body was thin, and some of the included reviews combine evidence from practising professionals 
(continuing professional development) with pre-service PD, which is often based in universities and less relevant to 
decisions that school leaders are making about developing their staff teams. Other evidence comes from reviews of 
approaches that have the potential to be delivered remotely, but the reviews include studies where the approach is 
deployed in face-to-face training (for example, reviews of video use in PD). Seven of the seventeen included reviews 
relate to school-based professionals, with the remaining evidence coming from welfare and public health PD, which 
may be less relevant to schools. Reviews also include studies from across the world, with USA the most common 
setting. 


The rapid nature of the review also means that it does not provide statistical analysis or comparisons between 
different approaches or reviews, but is instead a narrative summary of findings. One of the challenges this presents in 
interpreting the evidence is that it is difficult to make claims about the benefits of specific PD approaches. These 
limitations notwithstanding, this report aims to summarise evidence that may support schools and PD providers in 
making pragmatic decisions related to delivering professional development. 


Background and rationale 


As schools re-open and continue to operate in the context of Covid-19, the need for distancing may pose challenges 
to delivering professional development within school. Staff may be also unable to attend traditional face-to-face 
training outside school as national, local or personal circumstances may restrict movement, or schools may require 
teachers to be present in school. 


Schools and training providers, including EEF Research Schools, Promising Project providers and active trial 
providers, are considering ways in which they can use remote learning techniques to continue providing professional 
development for teachers and other members of staff. There is no evidence review of remote professional 
development in this context for them to draw upon. There is an urgent need for evidence on: 

- The qualities of successful remote learning techniques for professionals 

- How to successfully implement remote learning to facilitate adult professional development 

- How teachers and other school staff learn well when some or all training is remote 


For the purposes of this rapid evidence assessment, the scope is limited to direct professional development and thus 
discounts simple procedural training (that does not aim to change pedagogical approaches) as well as leadership 
programmes. While the population of interest is school-based professionals, the search scope was broadened to 
include practising professionals in similar spheres - welfare and public health - as we believe there are generalisable 
mechanisms for effectively supporting adults (online or otherwise) to change their practice that are relevant across 
sectors. The review sought to find the best evidence on which remote PD approaches are most likely to improve 
professional knowledge and behaviour and, ultimately, pupil (and equivalent service beneficiary) outcomes. 
Additionally, the review explored the characteristics of effective implementation of remote PD. 


Ideally a review of this scope would be conducted using an exhaustive systematic review. Given the urgent need for 
evidence now, a rapid evidence assessment was considered the most pragmatic solution as with two other recent 
reviews on remote learning (EEF, 2020a) and the attainment gap (EEF, 2020b). A comprehensive systematic review 
of professional development is due to be undertaken by the EEF in 2020-21, in order to publish a Guidance Report 
which will provide schools and PD providers with recommendations on how to effectively deliver professional 
development. 


Table 1: key definitions 


Professional development Activities, tools and resources for improving professional practice with the aim of 
improving outcomes for their service beneficiary (pupils in the case of school-based 
professionals). 

Remote delivery Methods of developing professionals that take place without face-to-face contact. 

Blended delivery Remote learning is used for some content delivery/instruction while other elements 


are delivered in person. This model can be contrasted against face-to-face models, 
where independent study and practice is likely part of the programme but all 
instruction is delivered face-to-face. 


Synchronous delivery Live sessions requiring the professional to attend sessions (remotely or in person) 
at specific times. Examples include live online lectures and coaching phone calls. 
Asynchronous delivery Learning sessions and resources that can be accessed at the professional’s 


discretion, as directed by a PD provider. Examples include a self-audit tool and 
pre-recorded online lectures. 

Self-directed learning Fixed tools, resources and activities that professionals can access independently of 
professional development programmes. Examples include teacher discussion 
forums and resource-sharing websites. 


Further detail on the background and rationale for this review, including a theory of change, can be found in the 
protocol here. 


Methods 


We undertook a rapid evidence assessment of existing systematic reviews and meta-analyses. We were guided by the 
Cochrane Collaboration Rapid Reviews Methods Group interim guidance on producing rapid reviews (Garrity et al. 
2020), as well as the Civil Service Rapid Evidence Assessment methodological guidance (Government Social Research 
Service, 2009) and the Cochrane Collaboration’s guidance on overviews of reviews (Pollock et al. 2020). A protocol for 
this Rapid Evidence Assessment, including theory of change, definition of professional development and more detail 
about the methodological approach taken, was published on the EEF website and is available here. 


Search criteria overview 


The scope was limited to systematic reviews and meta-analyses in order to respond quickly to the policy challenge of 
social distancing due to Covid-19. The review was limited to studies from 1995 or later, as technology-based 
approaches from pre-1995 are unlikely to be relevant to current approaches.' To be included in the REA, a study had 
to meet the following additional criteria: 


e Population: 
o Practicing professionals in education, welfare and public health (studies from any country) 
e = Intervention: 
o Direct PD where instruction is delivered remotely or partially remotely (‘blended’) 
e Comparison: 
o Any comparison including blended approaches, face-to-face PD, other remote PD or a no- 
intervention control group 
e Outcome: 
o Pupil or equivalent service beneficiary outcomes 
o Behaviour or knowledge change of the professional which is aimed at improving outcomes for pupils 
or relevant service beneficiaries 
o Other benefits for the professional such as changes to job satisfaction 
e Study design: 
o Meta-analyses or systematic reviews of remote PD effectiveness or implementation of remote PD 
e Other criteria: 
o Published since 1995 
o Published in English 
o Reviews published in peer-reviewed journals and grey literature 


How the review was conducted 


We undertook the following steps to produce the REA, as described in the protocol: 


1. 


2. 


Searched for relevant studies, including searching bibliographic databases and known sources of systematic 
reviews. 

Screened the search results for inclusion using the criteria described above in a two-stage process, first screening 
at title and abstract and then at full-text level. 

Extracted data from each included systematic review and meta-analysis, including information on methodological 
and substantive features, results, specifically pooled or individual effect sizes and associated confidence intervals 
where presented, and any information around barriers and facilitators to successful implementation of remote PD 
approaches. Where effect sizes are presented, these are as reported in the original review. 

Appraised each systematic review and meta-analysis for methodological quality, to make a judgement on how 
much confidence to place in the findings of each review. 

Summarised the findings of each included review, grouped by the broad topic they address. 


1 Although our search criteria allowed for studies from 1995, the oldest included review was published in 2008. 


Search Process Results 


Figure 1: PRISMA diagram of search process results 
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The initial searches returned 7444 references. After removing duplicates and screening on title and abstract?, 226 
references were labelled for inclusion. We were unable to retrieve 4 references, which meant that 222 were screened 
on full text. Of these 222, 17 reviews were included in this REA. All 17 had data extracted. The findings from the 17 
identified reviews are summarised below. 


? At title and abstract 1978 studies were screened using the priority screening function in EPPl-reviewer. Following 
100 excludes in a row using priority screening, and screening of a random sample of 30 of the remaining studies, the 
4458 remaining studies were excluded. 


Effectiveness of remote PD 


The approach 


This section examines the evidence base for fully remote PD compared to no PD, face-to-face instruction and 


blended PD approaches. 


Findings and implications: 


e Remote PD has the potential to improve professionals’ knowledge and skills and ultimately improve outcomes 
for pupils (and equivalent service beneficiaries in other professions). 

e Evidence is mixed on whether remote PD is more or less effective than face-to-face PD. Other design 
principles are likely to be more important to PD outcomes than whether delivery is face-to-face or remote. 


Relevant studies 


practice guidelines 


Review School Review focus Number of | What does the review say? 
review? studies 
Basma & Y Systematic review exploring the impact of Although not directly reported, 3 studies of 
Savage 2018 teacher PD on student reading achievement, 17 (3 remote PD are included, all with positive 
including 3 studies where remote video-based remote) effects that are comparable to the overall 
PD is compared to face-to-face PD effect size. 
Hines et al N A systematic review of interventions improving Same effect as face-to-face PD when 
‘ 10 ; 2 
2016 the research literacy of nurses filmed lectures are viewed remotely 
Hundey et al N Review of teaching-focused online and Online and blended mentorship 
2020 blended mentoring programmes for higher 17 programmes have the potential to promote 
education faculty teacher knowledge and skills 
Jackson et al N ‘ , There is evidence web-based PD has 
2018 ane en ibe eae PB itor 45 positive effects but face-to-face PD tends to 
P be more effective 
Kraft et al Y Meta-analysis of teacher coaching feo se : 
: : : ‘ Similar positive effect sizes for face-to-face 
on Toaaee OM Tectc eaenne —| agay | andonine PD out tis i a low conden 
progral y result as effects varied) 
Teaching Partner programme) 
Lynch et al Y While all types of PD improve outcomes for 
201? Meta-analysis of STEM teacher PD including a 95 (19 with Students, prograimmes.t winch participants 
: . : : completed a portion of the professional 
comparison of PD with any online elements to online : 
face-to-face models on outcomes for students | components) development onling nad weaker student 
outcomes, on average, as compared with 
those that did not include any online PD 
Peng & Yan N 7 ee 4 Pee Internet-based PD is more effective than no 
2017 Dea an intemnetpased earning 16 PD and as effective as face-to-face PD, 
P though effects vary between studies 
Semwal et al N Systematic review of digital education on All studies find that digital education is at 
2019 siniokiria cossation irianademenkior health 11 least as effective as traditional or usual 
ibs a 9 learning. Blended learning appears to offer 
P additional benefits. 
Singh et al N Systematic review of technology-based PD in : 
: , sie Technology-based PD is at least as 
2020 evidence-based treatment for clinicians 24 effective as manual-based or in-person PD 
(mostly mental health professionals) 
Tudor Car et N Digital education seems to be more 
al 2019 Systematic review of digital PD on clinical 17 effective than no intervention and at least 


as effective as traditional learning for 
improving participants’ knowledge 


How secure is the evidence? 


Not all of the studies included in the review directly made statements about the effectiveness of remote professional 
development: only ten of the seventeen studies are included in this section. The evidence is limited by the small 
number of rigorous studies that have taken place, particularly for the PD of teachers. It is also notable that the 
evidence between reviews is contradictory regarding the relative effectiveness of fully remote, blended and face-to- 


face PD. 


We conducted a quality assessment of each included systematic review and meta-analysis. The majority of the 
included reviews appraised the heterogeneity and quality or risk of bias of their included studies, with only one of the 


ten reviews considered a low confidence review. We therefore have reasonable confidence in the findings in this 
section. 


What does the evidence say? 


Remote PD is found to be effective in improving a range of outcomes, including practitioner knowledge and skills and 
improving outcomes for pupils or other service beneficiaries (Hines et al., 2016; Jackson et al., 2018; Kraft et al., 
2018; Lynch et al., 2019; Peng & Yan, 2017; Singh et al., 2020; Tudor Car et al., 2019). 


The evidence on the relative effectiveness of remote and face-to-face PD is less consistent. Several studies (Basma 
& Savage, 2018; Hines et al, 2016; Kraft et al, 2018; Peng & Yan, 2017; Semwal et al, 2019; Singh et al, 2020; Tudor 
Car et al, 2019) find remote learning approaches to be at least as effective as face-to-face PD. However, two other 
studies contradict these findings, with Lynch et al. (2019) finding any online delivery component is associated with 
less improvement in outcomes for pupils compared to those whose teachers have taken part in PD with no online 
component and Jackson et al. (2018) similarly finding that online PD is less effective than face-to-face PD. 


One review (Semwal et al., 2019) suggests blended PD may offer additional benefits to practitioner knowledge and 
skill over both fully online and face-to-face PD. This finding was based on 2 studies comparing blended to digital 
education and 3 studies comparing blended to usual learning. There was not enough evidence available to make 
assertions on the components of online learning associated with the greatest gains. 


Implications for PD in schools 


Three of the included reviews relate specifically to schools (Basma & Savage, 2018; Kraft et al, 2018; Lynch et al., 
2019). They reflect the mixed picture in the wider literature: remote PD can improve both the knowledge of school- 
based professionals and pupil outcomes but it is unclear whether such PD will be as effective as delivering PD face-to- 
face. It is likely that the relative effectiveness is more related to the content and context rather than the format of PD. In 
the present circumstances, remote delivery is an appropriate and potentially cost-effective way of providing PD for staff. 
Certainly, there does not seem to be strong evidence that school-based PD should be delayed until it is possible to 
deliver face-to-face. 


Relevant EEF-funded evaluations 


onebillion: 

The onebillion programme consists of two apps that are designed to support the acquisition of basic maths skills for children 
aged 3-6. It is an EEF ‘Promising Programme’ as children in the intervention group made 3 additional months’ progress 
compared to a comparison group in an EEF trial. As children work mostly independently on the apps, the training for Teaching 
Assistants (TAs) is a simple one-off session plus phone support. In the EEF efficacy trial, the initial training was provided face- 
to-face. 


In a pilot study moving training to a 100% online model, evaluators observed sessions to see how similarly the TAs ran the 
sessions compared to how the developer intended, turning the score for each observed session a ‘fidelity’ percentage rating. 
The programme was implemented with over 95% fidelity (similarity to how the developer intended) across both online and face- 
to-face conditions, with TAs in the online training group feeling confident and requiring less follow-up phone support than the 
comparison group. One (cover) TA who had not completed the training was observed at 70% fidelity, suggesting the training 


was effective in improving fidelity, though this finding should not be over-interpreted. Overall, the pilot suggests that straight- 
forward professional development can be transferred well to remote delivery. 


Nuffield Early Language Intervention (NELI): 

NELI is one of the most promising programmes the EEF has evaluated, following an impact of 3 months’ progress 
demonstrated in two separate EEF trials. TAs are trained to deliver this 20-week programme which aims to develop reception 
children’s early literacy skills. 


NELI can be characterised as a blended professional development programme. There is an initial 2-day face-to-face group 
training session and a half-day optional face-to-face workshop, complemented by webinar training, online support, phone 
support and an online forum. In our trial, most TAs accessed the remote support provided while only a third chose to attend the 
follow-up webinar. The results of this trial demonstrate the blended PD model can be beneficial for school professionals and 
their pupils. 


Synchronous and asynchronous remote PD 


The intervention 


Synchronous remote PD is live: the trainee accesses the PD at a fixed point in time. This could take the form of an 
online group discussion, a phone call with a mentor, or watching a live lecture. Asynchronous remote PD is accessed 
in the trainee’s own time, for example pre-recorded lectures, discussion boards or interactive games. For the 
purposes of this review, asynchronous PD is facilitated. This can be contrasted to self-directed PD, where a 
professional may, for example, read a book or online manual in their own time, without direction from an external 
tutor, training body or manager. 


Findings and implications 


e = There is insufficient evidence to conclude whether synchronous PD delivery is more effective than 
asynchronous. PD providers may wish to consider how to target the benefits of either approach to different 
aspects of PD. 

e Several studies identify that the targeted use of synchronous PD can provide benefits — for example, in 
teacher coaching. 

e There is also evidence of promise for asynchronous approaches — for example, through pairing asynchronous 
feedback with videos that allow teachers to review their own practice. 


Relevant studies 


Review School Review focus Number of | Findings 
review? studies 
N Review of teaching-focused online 


Online and blended synchronous mentoring 


RUndey et al anipspiendee Iner rong 17 rogrammes have the potential to increase teacher 
2020 programmes for higher education prog P f 
knowledge and skills 

faculty 
Jackson et al N Systematic review of web-based Both virtual classroom (synchronous) and serial 
2018 PD for behavioural health 45 instruction (asynchronous) designs increase 

providers practitioner knowledge 

Y Meta-analysis of teacher coaching 

programmes, including 13 virtual . P 

Kraft et al 2018 coaching programmes (10 of 60 (13 (Synchronous) virtual coaching supports teacher 


which relate to the My Teaching virtial) ain: Pup IOURROMES 


Partner programme) 


Different benefits attributed to each approach with 
Unclear stronger evidence for asynchronous video use to 
support classroom observation 


Marsh & Mitchell Review of the use of videos of 
2014 classrooms in teacher PD 


How secure is the evidence? 


Two reviews (Kraft et al., 2018; Jackson et al., 2018) measured the impact of a professional development approach 
on practitioner outcomes. The other two reviews provided a narrative synthesis of approaches. While the reviews 
reported findings for both synchronous and asynchronous PD, none directly compared the impact of similar 
approaches delivered both synchronously and asynchronously, and none of the reviews conducted meta-regression 
or moderator analysis allowing for comparable effect sizes between synchronous and asynchronous delivery. 


The inclusion criteria for two reviews may limit the relevance of the findings for schools and PD providers looking to 
support professional development while teachers are at home. In one case (Marsh & Mitchell, 2014), studies involving 
student teachers are combined with in-service teachers, which may mean the findings are not all relevant to in-service 
professionals. Furthermore, two reviews (Jackson et al., 2018; Marsh & Mitchell, 2014) focus on sessions being 
delivered remotely from the provider’s perspective, but it is not clear whether some studies are included in which the 
content is remote from learners but a group of learners access the PD together (for example teachers viewing a video 
together). 


We conducted a quality assessment of all included reviews. Only one of the reviews reported performing a 
comprehensive search of the evidence and using appropriate criteria to assess the quality and risk of bias in 
analysing the studies that were included in the review. We therefore do not have a clear understanding of the quality 
of the underlying research contributing to our conclusions. 


What does the evidence say? 


For individual PD, synchronous remote PD is found to be promising in both Hundey et al. (2020) and Kraft et al. 
(2018), which focus on one-to-one support from an expert professional in the form of mentoring and coaching 
respectively. In one paper that analysed an effect size (Kraft et al, 2018), the effect of virtual coaching on both teacher 
instruction and student achievement is comparable to face-to-face coaching, though this was not a secure finding. 
The lack of direct comparison with asynchronous one-to-one support means that it is not possible to identify to what 
extent synchronous delivery is contributing to the positive impact. 


For group PD, there is evidence of promise for both synchronous and asynchronous delivery. Jackson et al (2018) 
find 10 studies using the synchronous ‘virtual classroom’ design with a live, remote facilitator and 37 studies using 
asynchronous ‘serial instruction’ design, whereby learners undertake tutorials in a set order. Practitioner knowledge 
was positive across both, but with a wider range for asynchronous serial instruction (-1.70 to 4.70) compared to the 
synchronous virtual classroom (0.16 to 1.34). One study which compared this method to serial instruction reported 
higher levels of burnout for the virtual classroom design. The review posited that serial instruction methodology could 
save costs as it does not require instructor facilitation. 


Marsh & Mitchell (2014) focus on the use of video and discuss the potential benefits of both formats. Synchronous 
video viewing is seen as useful for teachers to view another teacher’s classroom activity in real time, which can be 
paired with opportunities for live group observation (note the review includes some studies which use video for face- 
to-face group sessions and for this finding does not specify whether group observation would be face-to-face or 
accessed remotely). Asynchronous video watching is described as particularly promising as it can allow teachers to 
view their own practice and can allow repetition and reflection over time, including being used to demonstrate specific 
learning points. 


Implications for PD in schools 


Two of the included reviews studied school practitioners specifically (Kraft et al, 2018; Marsh & Mitchell, 2014). There 
is tentative evidence that synchronous coaching may be as effective when delivered remotely and several studies 
support the use of video in both synchronous and asynchronous remote PD, particularly to view teacher classroom 
practice. 


Asynchronous delivery affords participants more flexibility in when they engage with the professional development. 
The tentative evidence available suggests this delivery style can be effective, so particularly as a response to 
restrictions caused by the Covid-19 pandemic and if professionals may have to access learning at home (with 
potential for temporary access problems or competing commitments), the inclusion of some asynchronous elements 
of delivery in training programmes might provide a pragmatic solution. 


Remote coaching, mentoring and expert support 


The approach 


As well as making decisions about the mix of synchronous and asynchronous format of remote PD, school leaders and 
providers have a decision to make that is both a format and a content question: should expert support be provided 
remotely to enhance PD? This section explores the evidence for different types of remote support provided for 
professionals, including mentoring, coaching and other expert support.? This can be provided on a one-to-one, one-to- 
many or many-to-one basis. 


Findings and implications 


e There is evidence that coaching and mentoring can improve skills and knowledge of professionals when 
delivered remotely and may reduce feelings of isolation in professionals 

e Remote or blended coaching, mentoring and expert support can be used to complement broader remote or 
blended PD programmes 


Relevant studies 


Review School review? Review focus Number of | Reported impact 
studies 

Gentry et al Y Review of implementation of Mentoring facilitated by technology has strong 

2008 technologically based mentoring and 14 potential for providing cost effective, 
instructional coaching interventions for sustained PD 
teachers 

Hundey etal N Review of teaching-focused online and Online and blended mentorship have the 

2020 y blended mentoring programmes for higher | 17 potential to promote teaching knowledge and 
education faculty skills 

Jackson et al N Ongoing support was associated with 

2018 Systematic review of web-based PD for 45 improvements in completion, knowledge, 
behavioural health providers attitudes, skill and fidelity but mixed 

comparison to no support 
Y Meta-analysis of teacher coaching Both online and face-to face coaching 

Kraft et al programmes, including 13 virtual coaching | 60 (13 | improve teacher practice and pupil outcomes; 

2018 programmes (10 of which relate to the My virtual) online coaching may be equally effective (low 
Teaching Partner programme) confidence result) 

: Y : ; aa Video viewing of classroom practice impacts 
Hanes 2018 pea rere a SUPRONIN 82 teacher cognition but only when combined 
: with high-quality support 

Singh et al N Systematic review of technology-based PD : F ‘ — 

2020 in evidence-based treatment for clinicians 24 Our ate calabiat pas 
(mostly mental health professionals) going SEPP 


How secure is the evidence? 


Six reviews explored the evidence around coaching, mentoring and the use of expert support. The oldest (Gentry et 
al., 2008) can be seen as a precursor to Kraft et al. (2018) as it calls for an exploration of the effect on pupils, which is 
studied in the more recent Kraft paper. The remaining reviews were all published within the last two years. 


We appraised the quality of each included review. Four of the reviews report the risk of bias for individual included 
studies and only one (Singh et al., 2020) analyses and reports results separately by risk of bias. This is a weakness in 
the literature. Overall, three of the six included reviews received a low quality rating. 


One of the papers (Kraft) calculates the impact of remote coaching approaches compared to face-to-face measures 
using pupil test results as an objective measure rather than teacher self-report gains. 


Only two other reviews (Jackson et al., 2018; Singh et al., 2020) use methodologies suitable for making claims about 
the comparative effectiveness of other expert support approaches and in both cases they did not find primary 
outcomes were improved compared to comparison groups with no expert support, thus we are unable to make 


3 There are several high-quality publications outlining the differences between coaching and mentoring and the ways different approaches can be 
implemented successfully. See for example the Chartered College of Teaching’s Teacher CPD Report, Chapter 4. Given the scarcity of evidence 
found on remote coaching and mentoring approaches in this review, findings are loosely combined to give a general overview of evidence to school 
leaders on expert support in remote PD. 
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assertions as to the effectiveness of expert support as part of PD programmes. It should also be noted that neither of 
these studies is based in the school context. Other reviews explore qualitative evidence suitable for making scoping 
comments but not effectiveness claims. 


This is an area where more research is crucial to understand the conditions under which expert support enhances 
professional learning, as noted in all included reviews. 


What does the evidence say? 


There is some evidence that coaching and mentoring approaches are feasible and can be effective when delivered 
remotely. Kraft et al. (2018) find a positive effect for online coaching based on 13 trials. The same paper observes no 
difference in effectiveness between virtual and face-to-face coaching, though these findings are tentative due to large 
standard errors, so impact results as compared to face-to-face coaching should be treated with caution. Similarly, 
studies included in Hundey et al. (2020) consistently report improvements in participants’ attitudes, beliefs, knowledge, 
skills or practices following participation in online or blended mentoring. 


The two reviews reporting effects of expert support compared to PD without this support (Jackson et al., 2018; Singh 
et al., 2020) both find a mix of outcomes in the studies reviewed, with some comparative studies finding no difference 
between groups with and without expert support. However, the authors stress in both cases that the intervention groups 
gained skill, knowledge and in some individual studies improved completion rates. It is suggested that the lack of 
comparative gain may lie in differences between the studies included within each review in: the intensity of the PD, 
frequency of sessions, duration of the programme and qualities of the expert practitioner. Both remain supportive of the 
inclusion of expert support in PD programmes. Singh states that ‘ongoing support may improve clinician knowledge and 
connection with peers and trainers’ while Jackson states: ‘in situations such as web-based PD where there is a lack of 
face-to-face contact with trainers and peers, the interaction that ongoing support emphasizes may be even more 
pertinent.’ 


Specific benefits are discussed: a common problem with coaching programmes is that the relationship can be hampered 
if the coach is also the teacher's ‘evaluator’ (line manager or induction tutor) whilst virtual coaching enables the 
relationship to be separate from the school context (Kraft et al., 2018). A recent review of higher education teaching 
staff (Hundey et al., 2020) finds that mentorship can reduce feelings of isolation or promote community building among 
professionals. 


Coaching, mentoring and expert support are often paired with other PD. One review notes ongoing support is 
associated with higher rates of course completion as well as higher fidelity (Jackson et al., 2018) while another (Kraft 
et al., 2018) finds that coaching programmes that are paired with group training have larger effects on pupil outcomes 
— though this latter finding includes studies of face-to-face coaching so might have less relevance to remote PD. 


Implications for PD in schools 


Coaching, mentoring and expert support are likely to be a feature of school-based PD and can be delivered remotely 
or partially remotely. Remote coaching and mentoring may enable school leaders to provide coaches or mentors who 
sit outside of the staff member’s performance management processes. PD providers will need to consider the most 
effective and efficient use of expert support based on the content of their PD programme. 


Evidence on common approaches 


This section explores the review evidence on several features of remotely delivered PD programmes that were 
commonly mentioned in the literature: interactive content, use of videos, and collaboration with colleagues. 


1. Interactive content 
Findings and implications 


e Increasing interactivity tends to improve completion rates of PD and may increase knowledge and skills 

e More interactive content tends to increase the time practitioners take to complete PD, which may be due to 
closer engagement with the materials 

e Spaced education approaches are particularly promising and may offer a low-cost way of enabling ongoing 
interaction with PD content 


Interactive content 


Review School Review focus Number of What does the review say? 
review? studies 

Cho et al 2020 Y Systematic literature review describing 22 (16 Computerised and mixed-reality 
technology's roles in classroom relating to interactive simulations can support 
management and school discipline, including technology classroom management skills by 
nine studies of PD using video and 5 studies in PD) increasing realism with low cost and risk 
using interactive simulations. to students 

Cook et al 2010 N A range of features associated with 

‘ i , ; making the content interactive 

Systematic review and meta-analysis of time 20 (animations, short video clips, audio 


and learning efficiency in internet-based PD narration) improve knowledge gains and 


increase PD time length 
Hines et al 2016 N Increased interactivity of the intervention 
A systematic review of interventions improves effectiveness as does using 
improving the research literacy of nurses educational or behaviour change theory 
to develop interventions 
Non-interactive courses are more likely 
17 to have high attrition and a spaced 
approach shows promise 


Tudor Car et al 2019 , Systematic review of digital PD on clinical 


practice guidelines 


How secure is the evidence? 


We appraised the quality of each included review. Three of the four reviews are well-conducted systematic reviews 
with few limitations (Cook et al., 2010; Hines et al., 2016; Tudor Car et al., 2019). A final review has unclear criteria 
applied for establishing the quality of included studies (Cho et al., 2020). Two of the reviews (Cook et al., 2010; Cho 
et al., 2020) include studies concerning pre-service professionals, which makes it more difficult to generalise the 
findings to in-service professionals. 


The only review relating to schools has a low confidence rating. Therefore, while the evidence in this area is relatively 
strong, a potential weakness is that this evidence may not transfer fully to the school setting. 


What does the evidence say? 


One review finds that trainees had higher knowledge scores in one study each where remote PD includes audio 
narration, video clips in short chunks or animations but that each of these additions mean professionals take longer to 
complete the PD; conversely, a study incorporating self-assessment questions before or after content delivery does 
not increase knowledge or time spent learning (Cook et al., 2010). It is concluded that internet-based instruction and 
non-computer instruction require similar time as instructional strategies which increase interactivity typically prolong 
learning time while enhancing learning outcomes. Supporting this finding, Hines et al. (2016) find increased 
interactivity is associated with better outcomes for nurses engaging with research, as is improving engagement 
through careful design of content based on educational or behavioural change theory. Cho et al. (2020) find that 
online classroom management PD can be enhanced by the inclusion of interactive content simulating the classroom 
experience. 


Conversely, Tudor Car et al. (2019) find mixed results on whether interactive content improves attitudes, knowledge 


or skills. The heterogeneity is explored and it is discovered that completely non-interactive (often self-directed) 
courses such as website content were more likely to suffer attrition. While performance feedback was not associated 
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with significant comparative gains, three studies using spaced education delivered via regular email surveys or online 
simulation reported large beneficial effects. 


One isolated but promising example is an adaptive design, whereby programming is used to change which content 
participants are shown based on their answers to questions. In a single study, it took participants 18% less time than 
a comparison group following the same curriculum without adaptive content while gaining comparable knowledge test 
scores (Cook et al., 2010). This approach may have limited transferability to school-based PD and is known to be 
difficult to design well but is worth consideration for topics where it may be relevant. 


Implications for PD in schools 


Increasing the opportunities for professionals to engage with content through making it more interactive is likely to 
increase the time taken for staff to complete the PD course and is likely to see benefits in terms of knowledge and 
skills gained and completion rates. PD providers may wish to consider using interactive simulations to cover practical 
content that benefits from repetitive practice, such as behaviour management. 


2. Use of video 


Video technology can be used synchronously or asynchronously to enable practitioners to view and reflect on their own 
practice, to view other practitioners’ practice from a physical distance or at a later point in time and to allow live and pre- 
recorded PD sessions to be shared with remote trainees. 


Findings and implications 


e The use of videos in PD generally increases the time required for trainees but is associated with gains in 
practitioner knowledge and pupil outcomes, suggesting the additional time might be spent engaging more 
closely with the content. 

e Video can be particularly helpful to enable teaching staff to review their own and reflect on others’ actions in 
the classroom 

e Video viewing is unlikely to be impactful in isolation and should instead be paired with monitoring tools and 
frameworks, other activities such as role playing and practice or discussion with other professionals. 


Use of video 
Review School review? Review focus Number of What does the review say? 
studies 
Basma & Y Systematic review exploring the impact of 
Savage 2018 teacher PD on student reading 17 (3 Students whose teachers receive remote PD 
achievement, including 3 studies where remote through video exemplars show better results 
remote video-based PD is compared to PD) than control groups 
face-to-face PD 
Cho et al Y Systematic literature review describing : : ‘ 
; : 22 (16 Video may help deliver content or provide 
2020 eg dea a ear a relating to learners with a sense of realism (by showing 
‘ 9 . eipine; technology a real or role-played classroom) when 
including 9 studies of PD using video and in PD) coupled with comprehensive learning aides 
5 studies using interactive simulations 
Cook et al., N 
2010 Systematic review and meta-analysis of The use of short video clips improves 
time and learning efficiency in internet- 20 knowledge outcomes but means PD takes 
based PD longer 
Y Meta-analysis of teacher coaching Studies with video libraries of other teachers 
; : : ; classroom instruction used alongside 
Kraft et al programmes, including 13 virtual coaching 60 (13 : , 
2018 rogrammes (10 of which relate to the M virtual) Spec Hing (N=) Naas IOWer Imp Aor ian 
an Partner programme) y those which did not (but this finding includes 
9 preg face-to-face coaching) 
: Vid f cl 
. . : sh ideo viewing of classroom practice impacts 
Meee 2018 ae ee torsupporn cane teacher cognition but only when combined 
: with high-quality support 
Y Video viewing and discussion can potentially 
Marsh & Review of the use of videos of classrooms Uineiéar enhance teacher learning but is dependent on 
Mitchell 2014 in teacher PD the interactions and reflections amongst 
learners 
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How secure is the evidence? 


Of the five reviews regarding the use of video technology in remote PD, three had methodological weaknesses that 
mean interpreting effectiveness of using video should be approached with caution. The highest quality rating was 
awarded only to the review not based in education (Cook et al., 2010). Two of the reviews (Cook et al., 2010; Cho et 
al., 2020) include studies concerning pre-service professionals, which makes it more difficult to generalise the findings 
to in-service professionals. There is, however, consistency between the five reviews, which all find evidence of 
promise that videos can be used to support PD. 


In contrast, one well-conducted school-focused review (Kraft et al., 2018) compares coaching programmes. 
Programmes where video libraries of other teachers are used to support coaching conversations are compared 
against coaching programmes which do not use video libraries. The findings in this review contradict those of the five 
included studies, with a smaller positive effect on pupil outcomes seen for coaching programmes which include video 
libraries. The review does not separate out findings for remote and face-to-face PD for this analysis so it is not 
possible to determine the finding for remote PD only, however it is a reason to be cautious of the positive findings in 
the other five reviews. 


What does the evidence say? 


On the use of videos, five of six found that the use of video could enhance learning in PD. Video was most often used 
to review practice, either by viewing other professionals or watching back footage of a teacher’s own classroom. One 
study specifically found that short video clips enhanced practitioner knowledge while increasing the time spent 
learning (Cook et al., 2010). One review (Kraft at al., 2018) finds teacher coaching programmes have a smaller 
benefit for pupil outcomes when video libraries are utilised, but this finding is not related only to remote PD. 


Implications for PD in schools 


Videos were often used in teacher PD to exemplify teaching practice or enable the teacher to reflect on their own 
actions in the classroom. Three reviews (Cho et al., 2020; Major & Watson, 2018; Marsh & Mitchell, 2014) 
emphasised the need to pair videos with other support: discussion with peers and experts or comprehensive learning 
aides. The positive findings relating to using video were found for synchronous and asynchronous PD, not for content 
professionals could use in an independent, self-directed manner, so PD providers would be advised to consider the 
purpose and use of video content before its creation. 


3. Collaboration 


Studies relating to collaboration emphasised the social aspect of learning between professionals. The collaborative 
approach could be taken with a peer, within a group or with a mentor. 


Findings and implications 


e Collaboration between colleagues may improve PD outcomes through enabling reflective practice and 
collective problem-solving 

e Training may be required to ensure personal and professional boundaries are clear in the remote context 

e Colleagues within the same setting may be able to enhance the benefits of remote or blended PD 
programmes by holding implementation workshops within their own setting 


Relevant studies 
Collaboration 


Review School A Number of 5 
reviows? Review focus ties Reported impact 
Barry et al., 2017 N Review of communities of practice (CoPs) : , 
for occupational therapists, including a 5 (2 remote) Social context of leaming appears to 


comparison with one online CoP facilitate knowledge acquisition 


Chan & Leung, 2018 N Social networks can have benefits in a 
33 PD context but professionals require 


training in appropriate use 


Systematic review of use of social network 
sites among health professionals 


Gentry et al 2008 Y Review of implementation of 
technologically-based mentoring and 
instructional coaching interventions for 


Collaborative style of mentoring 
14 supports better outcomes than acting as 
supervisory ‘experts’ 


teachers 
N Review of teaching-focused online and Online and blended mentorship have 
Hundey et al 2020 blended mentoring programmes for higher 17 the potential to promote teaching 
education faculty knowledge and skills 


Lynch et al. 2019 Y Programmes that included 
Meta-analysis of STEM teacher PD troubleshooting and classroom 


including a comparison of PD with any ap ail implementation discussion were 
online elements to face-to-face models on components) associated with higher student gains 


outcomes for students (but this finding includes face-to-face 


Y . : Ae . Video viewing of classroom practice 
Major & Watson, 2018 ores ei se of video to support in-service et impacts teacher cognition but only when 
combined with high-quality support 
Y Video viewing and discussion can 
: Review of the use of videos of classrooms potentially enhance teacher learning but 
MATS LeNMCH Neo a in teacher PD riceal is dependent on the interactions and 


reflections amongst learners 


How secure is the evidence? 


Seven reviews had findings relating to collaboration, but most had methodological weaknesses: five of the reviews 
were unclear in their methods for including studies and one mentioned the lack of objective measures of teacher 
knowledge gains as a drawback in the included literature. Although the included reviews correspond, all citing positive 
benefits of collaboration between colleagues, most of the studies did not meet our criteria for rigour in reporting 
standards, usually due to lack of clarity on their criteria for inclusion of studies in their analysis. 


Only one review included outcomes relating to pupils or equivalent service beneficiaries and this review combined 
findings for online learning programmes with face-to-face PD (Lynch et al., 2019). Another review comparing remote 
and face-to-face communities of practice found the remote group gained fewer benefits than the comparison, though 
this finding relied on a single study and was not conducted with school staff (Barry et al., 2017). It is therefore difficult 
to distinguish whether collaboration has an impact on practitioner outcomes in remote PD or is merely common and 
well-regarded practice. 


What does the evidence say? 


Collaboration between colleagues is associated with practitioner knowledge gain (Barry et al., 2017; Chan & Leung, 
2018; Hundey et al., 2020; Marsh & Mitchell, 2014) and improved pupil outcomes (Lynch et al., 2019) as well as 
distinctive benefits in enabling collective problem-solving (Lynch et al., 2019; Major & Watson, 2018; Marsh & 
Mitchell, 2014) and providing social professional connections (Barry et al., 2017; Chan & Leung, 2018). 


Several different approaches to remote collaboration are explored in the literature, including peer collaboration (such 
as through social networking platforms and discussion forums) and collaboration with a mentor or coach via email, 
phone or online platforms (discussed in the section above). All appear to have the potential to support practitioner 
learning through self-reflection, though it should be noted the evidence is tentative. Evidence from one study suggests 
communities of practice have fewer benefits to participants’ knowledge when delivered remotely, although they bring 
additional gains in connecting professionals at a greater geographical distance (Barry et al., 2017). 


There were several findings relating to implementation in the literature: 
-  PDonsecure and appropriate use of online forums and social media is a pre-requisite for their inclusion in 
PD (Chan & Leung, 2018; Gentry et al., 2008) 
- Social media tended to be more highly valued by younger and less experienced staff, who were more likely to 
cite gaining professional connections as a benefit (Chan & Leung, 2018) 
- Protected collaboration time should be provided by the school (Gentry et al., 2008; Hundey et al., 2020) 


Implications for PD in schools 


PD providers may choose to incorporate collaborative elements into PD for reasons other than improving teacher 
knowledge or pupil outcomes. Other benefits might include strengthening professional networks and ensuring 
teachers feel well-supported. 


One review (Lynch et al., 2019) found that in-school collaboration supported better pupil outcomes (in a combination 
of face-to-face PD and PD with some online elements). Although the review does not suggest combining in-school 
collaboration with remote PD, if teachers are unable to attend face-to-face PD but can speak to colleagues in school, 
a blended approach combining remote PD with in-school collaboration may be a pragmatic solution for school leaders 
to try. 


PD providers may wish to consider which remote elements of their PD require collaboration between whom and 
ensure the timing or structure of these elements allows all participants to attend, whether this is synchronous or 
asynchronous. 


Supportive conditions 


This section explores the conditions needed in the work environment to enable professionals to engage successfully 
with remote PD. 


Findings and implications 


e School leaders have a critical role to play in ensuring enabling conditions are provided for coaching and 
mentoring relationships to be successful 

e Supportive conditions for successful PD include access to and training on appropriate technology; protected 
time to engage with sessions; clarifying the purpose of the relationship, roles and expectations 


Relevant studies 


Review School review? Review focus Number of | Reported impact 

studies 
Chan & | N , P ; Social networks can have benefits in a PD 
Leung, 2018 Bye eretG Veil Use Ole belal HeIwole 33 context but professionals require training in 


sites among health professionals appropriate use 


Gentry et al Y Review of implementation of Mentoring facilitated by technology has strong 
2008 technologically based mentoring and 14 potential but requires adequate, user-friendly 
instructional coaching interventions for technology, time in workday for teachers to 
teachers engage and supportive culture 
N Successful implementation of online and 
H Review of teaching-focused online and blended mentorship depends on dedicated 
undey et al : : : : , 
2020 blended mentoring programmes for higher | 17 human and material resources including 
education faculty support from senior administrators, alongside 
well-functioning technology 
Y No clear conclusion on how learning 
: : environments should be constructed, but 
Caio 14 lA a Usp Ot yideds oliclassteots Unclear relevance and technical quality of the video is 
important in effectiveness of video-viewing to 
enhance PD 


How secure is the evidence? 


Four reviews explored implementation questions regarding the supportive conditions required for professionals to be 
able to engage successfully with their remote PD. Each review has a specific PD focus (social networks, coaching, 
mentoring and video use) so their findings may not generalise to all PD programmes. 


We assessed the included reviews for quality. Three of the four reviews were rated as low quality, one (Gentry et al., 
2008) medium quality, due to a lack of clarity regarding their inclusion criteria for studies or quality assessment of 
studies. Despite the low scoring, the included reviews provide a narrative discussion rather than making causal 
claims, so the findings can still be regarded as valid for consideration alongside professional judgement. 


What does the evidence say? 


The included reviews highlight several factors that may affect successful implementation: 
- technology is available, of a reasonable quality and user-friendly (Gentry et al., 2008; Hundey et al., 2020; 


Marsh & Mitchell, 2014); 

- professionals are adequately trained to use platforms required for the approach (Chan & Leung, 2018; Gentry 
et al., 2008); 

- leadership teams support the PD, including by allowing time in work hours for the sessions to happen (Gentry 
et al., 2008; Hundey et al, 2020); 

- Professionals have some flexibility about when and how PD takes place (Hundey et al., 2020); 

- expectations of the sessions are clear, PD objectives are shared and remain the focus throughout the PD 
(Gentry et al., 2008; Hundey et al, 2020) 


Implications for PD in schools 
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PD providers might be able to support professionals with some of these enabling conditions, but the weight of 
responsibility falls on school leaders and managers. 


When commissioning external remote PD, particularly if this is a new approach, school leaders might find it useful to 
review the list above to ensure they have all the supportive conditions in place to enable the professional(s) to access 
the PD effectively. A conversation with the school leader or their presence at information-giving sessions may support 
external PD providers to ensure all professionals have appropriate enabling conditions met. 


When designing internal PD, school leaders may wish to consider delaying implementation of this PD until barriers 
can be removed, such as adapting an individual teacher's heavy timetable or updating the school’s computer 
operating systems. 


For more guidance on implementing change in schools, see the EEF’s Guide to Implementation. 


Meta-analyses and systematic reviews included in the rapid evidence 


assessment 
Pooled 
: : effect (95% 
Author Year Title Intervention type Buntidonee Outcome 
interval) 
Barry, M; Kuijer- 2017 
Siebelink, .; 
Nieuwenhuis, L; Communities of practice: A means to support N/A Outcomes not defined 
Scherpbier-de Haan, occupational therapists’ continuing Communities of 
N professional development. A literature review Practice 
2018 Teacher Professional Development and 0.2254 
Student Literacy Growth: A Systematic (0.064- Student literacy 
Basma, B; Savage, R Review and Meta-Analysis Video exemplars 0.385) 
2018 Use of Social Network Sites for 
Chan, Windy SY; Communication Among Health Social media N/A Outcomes not defined 
Leung, Angela YM Professionals: Systematic Review networking 
2020 Videos, Various, primarily 
Cho, V; Cumings The past and future technology in classroom interactive NA teacher classroom 
Mansfield, K; management and school discipline: A simulations, management knowledge 
Claughton, J systematic review databases and skills 
2010 Time and learning efficiency in Internet- , ae : 
Cook, DA; Levinson based learning: a systematic review and Internet-based N/A fr aeten eae 
AJ; Garside, S meta-analysis. learning 
2008 Teacher attitudes, 
Gentry, LB; Denton, Technologically-based mentoring provided to Technology- N/A beliefs, practices; 
CA; Kurz, T teachers: A synthesis of the literature based mentoring student achievement 
Hines, S; 2016 The Effectiveness of interventions for Research 
Ramsbotham, J; improving the research literacy of nurses: a literacy N/A Research literacy 
Coyer, F systematic review interventions 
Hundey, B; Anstey, L; 2020 Mentoring faculty online: a literature review : : 
Cruickshank, H; and recommendations for web-based Online N/A ae i eae , 
Watson, GPL programs mentoring 9 
Jackson CB; Quetsch 2018 a 
LB; Brabson LA; Web-Based Training Methods for Behavioral Web based N/A bigot a 
Herschell AD Health Providers: A Systematic Review training methods 
2018 The Effect of Teacher Coaching on 0.49 SD® 
Kraft, MA; Blazar, D; Instruction and Achievement: A Meta- Teacher (95% Cl not Teacher instruction 
Hogan, D Analysis of the Causal Evidence coaching reported) 
Lynch, K; Hill, HC; 2019 Strengthening the Research Base That STEM teacher 0.216 
Gonzalez KE; Pollard Informs STEM Instructional Improvement development (95% Cl not Student achievement 
Cc Efforts: A Meta-Analysis programmes reported) 
2018 Using Video to Support In-Service Teacher ss 
Professional Development: The State of the N/A Lion oe ale 
Major, L; Watson, S Field, Limitations and Possibilities Video-based PD MEE ene eee 
2014 The Role of Video in Teacher Professional NA Teacher professional 
Marsh, B; Mitchell, N Development Video-based PD knowledge 
2017 Effect of Internet-Based Learning in Public 0.437 
Health Training: An Exploratory Meta- Internet-based . Professional knowledge 
. 2 : (0.15, 0.72) 
Peng, Y; Yan, W Analysis learning 
2019 Digital Education for Health Professions on is 
Semwal, M; Whiting, Smoking Cessation Management: N/A Practitioner knowledge, 
seg See as eae : : at skills, attitudes, and 
P; Bajpai R; Bajpai S; Systematic Review by the Digital Health satisfaction 
Kyaw BM; Tudor Car L Education Collaboration Digital Education 
2020 Using Technology to Train Clinicians in Teachnology- ney 
Singh T.; Reyes- Evidence-Based Treatment: A Systematic based PD on N/A Clinician knowledge and 
, : : skills 
Portillo J.A. Review evidence use 
2019 Health professions digital education on Health professionals’ 
Tudor Car L; Soong A; clinical practice guidelines: a systematic N/A knowledge, skills, 
Kyaw BM; Chua KL; review by Digital Health Education Digital education attitudes, and 
Low-Beer N; Majeed A collaboration. on evidence use satisfaction 


“Effect of PD on student literacy. Results for remote PD are not analysed separately. 
5 Effect of all coaching on teacher instruction compared to no intervention (the estimated effect on student achievement is 0.18 SD). Their meta- 
regression estimates that virtual coaching reduces the pooled effect size on teacher instruction by -0.16 SD, compared to programmes without 
virtual coaching, however the author states: “We do not find any significant difference in effect sizes for coaching programs that were delivered in 
person or virtually, although our standard errors are too large to rule out even moderately sized differences”. 
® This is the effect size for teacher PD programmes, compared to no intervention. The meta-regression estimates that PD with any online 
components reduces the pooled effect size by -0.16 SD, compared to programmes with no online components. 
’ This is the effect size reported for professionals (separated from studies of students), compared to no intervention. The main effect size, 
combining student professionals) is 0.36 (-0.10,0.83). 
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Conclusion 


1. Professional development can be supported effectively remotely 


School professionals can gain knowledge and skills through remote PD, ultimately leading to gains in pupil 
outcomes. 


Evidence is mixed as to whether remotely delivered PD is more or less effective than face-to-face PD, but specific 
benefits include lower costs and time incurred through travel. Other design principles are likely to be more important 
to PD outcomes than whether delivery is face-to-face or remote. There is no strong evidence that school-based PD 
should be delayed until it can be delivered face to face. 


Blending synchronous and asynchronous delivery and blending face-to-face and remote learning may offer specific 
benefits by enabling trainees to feel part of a community while retaining travel cost savings, flexibility and social 
distance during remote and asynchronous elements. 


2. Remote coaching, mentoring and expert support can be effective alone or as part of broader PD 
programmes 


Coaching and mentoring can improve skills and knowledge of professionals when delivered remotely and may 
reduce feelings of isolation in professionals. 


Remote or blended coaching, mentoring and expert support can be used to complement broader remote or 
blended PD programmes. 


3. The use of video can enhance remote PD 


The use of video is identified across a number of reviews as a particularly effective element of PD that enables 
teaching staff to review their own and reflect on others’ actions in the classroom. The targeted use of videos 
generally increases the time trainees take to complete PD but is associated with gains in practitioner knowledge 
and pupil outcomes, suggesting the additional time might be spent engaging more closely with the content. 


Video viewing is unlikely to be impactful in isolation and should instead be paired with other learning resources 
such as viewing guides or discussion with other professionals. One contradictory finding (though not limited to 
remote PD) suggests caution and monitoring is necessary when using video resources to support coaching 
conversations. 


4. Interactive content and opportunities for collaboration hold promise for remote professional 
development 


More interactive content tends to increase the time practitioners take to complete PD and is associated with better 
completion rates, knowledge and skill acquisition. Soaced education approaches (such as regular email surveys) 
could be particularly promising and may offer a low-cost way of enabling ongoing interaction with PD content. 
Conversely, barriers such as information being difficult to access can have a detrimental effect on user 
engagement. 


Collaboration between colleagues may also improve PD outcomes through enabling reflective practice and 
collective problem-solving. For example, PD providers may include peer small-group discussion sessions 
following completion of individual tasks. 


5. Remote professional development requires supportive school conditions (Support from leaders, 
protected time, tech-specific training, platform ease of access) 


School leaders have a critical role to play in ensuring enabling conditions are provided for remote PD to be 
successful. They can support staff to prioritise their PD by creating protected time within the working day for staff 
to engage with PD sessions or materials. 


Schools should ensure staff have access to technology required for their PD and appropriate training in order to 
access this safely, efficiently and appropriately. PD outcomes are strengthened where the purpose of the PD, 
roles and expectations are clear; this may require co-ordination from the PD provider, trainee, and school 
leadership team. 
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The evidence in this review is drawn from diverse contexts that do not closely parallel the circumstances facing schools 
responding to Covid-19 in 2020. In all cases, it is important for teachers and school leaders to use their judgement in 


determining appropriate professional development approaches for staff. 


Research questions 


The systematic search retrieved 17 systematic reviews and meta-analyses. 


1. What evidence is available on the effectiveness of remote PD approaches, compared to face-to-face, 
blended approaches and other remote approaches? 


There is consistent evidence that remote PD can support improvement in professionals’ knowledge and skills 
and in outcomes for pupils. This finding comes from reviews of school-based studies (n=4) and those from other 
fields (n=7). Of the eleven studies that directly compare remote learning to face-to-face or blended learning 
approaches, results are mixed as to which approach is more effective. 


The evidence is spread across a wide range of different interventions and study designs. This review therefore 
does not attempt to measure the impact of remote or blended PD in comparison to face-to-face PD overall, but 
instead summarises the impact of different reviews and approaches thematically. 


2. Does some face-to-face contact moderate outcomes in remote PD approaches? How do different 
remote PD approaches moderate outcomes? 


The methodological limitations of this review of existing systematic reviews and meta-analyses means that it is 
not possible to systematically examine differential impacts of different approaches. There are several trends 
within the reviews in which research teams have highlighted particularly promising approaches, or conducted 
moderator analysis that indicate differential effects. 


- Coaching, mentoring and expert support can be delivered remotely with evidence that skills and 
knowledge can be improved through these interventions delivered alone or as part of broader PD 
programmes 


- Video use in remote PD is associated with increased reflective practice, knowledge gain and pupil 
outcomes when paired with discussion or learning resources 


- Spaced learning approaches (such as regular email surveys of PD participants) may offer particular 
promise as a low-cost methodology which could enhance remote PD outcomes 


3. What can we learn from existing systematic reviews and meta-analyses about the characteristics of 
effective remote PD implementation? 


Evidence was mixed and inconclusive between synchronous, asynchronous or mixed instruction, with reviews 
identifying promise in each approach, although completion rates were likely to be lowest in fully asynchronous 
delivery. 


There were several features identified across multiple reviews as supporting effective remote PD 
implementation: 


- Supportive conditions for remote PD to take place included protecting time for the PD to take place and 
providing PD relating to the technology used in the intervention, including relating to professional 
behaviour online 


- Opportunities to collaborate with colleagues was cited as promising in several reviews, though high 
quality evidence was lacking in this area 


- There was strong evidence that engaging and interactive content supports knowledge gain in 
professionals, though the strongest evidence comes from outside the school context 
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Limitations 
This review has been limited by a number of factors. 
Umbrella review methodology 


In order to complete this review quickly and to respond to schools’ and school PD providers’ need for evidence on 
remote PD, this review was limited to existing systematic reviews and meta-analyses. This means that we are unable 
to perform moderator analysis on specific types of remote PD and cannot apply quality criteria to individual studies. The 
limitations of this methodology mean that we have not provided an overall estimate of impact for remote PD, or for any 
of the specific approaches through which remote PD may be delivered. 


Limitations in the evidence-base on remote PD 


There were very few reviews included in the final report. From our background research, we concluded that we would 
need to use findings from across the world and from a wider field than just education in order to find enough evidence, 
and thus extended the criteria to include professionals in welfare and public health fields globally. While the rationale 
for including these professionals was sound, in reality it was difficult to categorise reviews, particularly into the included 
field of public health as opposed to the excluded field of healthcare, particularly as several reviews included studies 
working across both. Some included reviews could be characterised as serving healthcare professionals in public health 
roles. 


Many of the reviews in the evidence base failed to find comparative evidence that related to knowledge gains, behaviour 
changes or other measures relating to professionals or relating to pupil outcomes (or equivalent service beneficiaries 
in other professions). Many reviews that included studies from relevant professions also included studies of pre-service 
professionals and these were usually greater in number. In these combined reviews, it was often unclear in moderator 
analysis how many studies from in-service professionals featured in the analysis or unclear whether there were 
differential effects between these two distinct groups. Other approaches, such as using video or simulation to support 
PD, were more prevalent among reviews of in-service professionals but did not always take place remotely. 


Quality of review methods within education 


One of the interesting findings of the review was the low quality of review methods amongst the retrieved studies. We 
rated the quality of each of the reviews using a number of criteria (the full rating system is available in the protocol 
here). Many of the studies did not apply a risk of bias assessment for the included studies. This meant that 7 of the 17 
included reviews were rated as “low” in the quality assessment. While some reviews did explore methodological causes 
for heterogeneity — for example, sample size or type of measure — the fact that few reviews assessed the quality of the 
individual studies included in the reviews is a severe limitation to the quality of the reviews summarised in this rapid 
evidence assessment. 


Future research 


A technical write up of the rapid evidence assessment will be published in the future. Further research is recommended 
comparing face-to-face, online and blended delivery of the same PD programme for school professionals. 
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Additional resources 


The Education Endowment Foundation has created a number of resources that are relevant to supporting learners 
during the Covid-19 outbreak. All resources can be found here. 


Some of the resources that directly relate to the findings of the remote professional development rapid evidence 
assessment are detailed below: 


priority issues based on the best available evidence. 


Resource Description Link 

Remote Investigates methods that schools could use to https://educationendowmentfoundation.org 

learning rapid | support remote learning during school closures .uk/public/files/Publications/Covid- 

evidence caused by the 2020 coronavirus pandemic (Covid- 19 Resources/Remote_learning_evidence_ 

assessment 19), based on best available evidence. review/Remote_Learning_Rapid_Evidence_ 
Assessment.pdf 

Covid-19 A range of online EEF resources to support schools | https://educationendowmentfoundation.org 

resources to respond to the current challenges -uk/covid-19-resources/ 

Attainment Examines the potential impact of school closures on | https://educationendowmentfoundation.org 

gap rapid the attainment gap, based on a systematic search of | .uk/public/files/EEF_(2020) - 

evidence existing literature. Impact_of_School_Closures_on_the_Aittai 

assessment nment_Gap.pdf 

Guidance EEF guidance reports provide clear and actionable https://educationendowmentfoundation.org 

reports recommendations for teachers on a range of high- -uk/tools/guidance-reports/ 


Teaching and 
Learning 
Toolkit 


The Teaching and Learning Toolkit provides an 
accessible summary of the evidence across 35 
different approaches aimed at improving pupil 
outcomes 


https://educationendowmentfoundation.org 
.uk/evidence-summaries/teaching-learning- 
toolkit/ 


Metacognition 


Toolkit summary of metacognition and self- 


https://educationendowmentfoundation.org 


Clearinghouse 
list of studies 


learning approaches, identified by the What Works 
Clearinghouse 


and self- regulation approaches -uk/evidence-summaries/teaching-learning- 
regulation toolkit/meta-cognition-and-self-regulation/ 
Digital Toolkit summary of digital technology approaches https://educationendowmentfoundation.org 
technology .uk/evidence-summaries/teaching-learning- 
toolkit/digital-technology/ 

EEF funded This is the full list of evaluations that have been https://educationendowmentfoundation.org 
evaluations funded by the EEF. .uk/projects-and-evaluation/projects/ 

What Works A list of studies that examine the impact of remote https://ies.ed.gov/ncee/wwe/distancelearnin 


gstudy 
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